United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.usplo.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 



10/772,028 02/03/2004 Paul Kjuszewski 40128/01301 5341 



30636 7590 11/28/2007 

FAY KAPLUN & MARCIN, LLP 
150 BROADWAY, SUITE 702 
NEW YORK, NY 10038 



EXAMINER 



REPKO, JASON MICHAEL 



ART UNIT 



2628 



PAPER NUMBER 



MAIL DATE 



11/28/2007 



DELIVERY MODE 



PAPER 



Please find below and/or attached an Office communication concerning this application or proceeding. 



The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



Office Action Summary 


Application No. 

10/772,028 


Applicant(s) 

KRUSZEWSKI ET AL. 


examiner 

Jason M. Repko 


Art Unit 

2628 





~ The MAILING DATE of this communication appears on the cover sheet with the correspondency address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 17 September 2007 . 
2a)IEl This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-49 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-49 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10)S The drawing(s) filed on 03 February 2004 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 

Paper No(s)/Mail Date . 



4) C] Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) CD Notice of Informal Patent Application 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20071123 



Application/Control Number: 

10/772,028 

Art Unit: 2628 



Page 2 



DETAILED ACTION 
Claim Rejections - 35 USC §103 

1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the 
various claims was commonly owned at the time any inventions covered therein were 
made absent any evidence to the contrary. Applicant is advised of the obligation under 
37 CFR 1 .56 to point out the inventor and invention dates of each claim that was not 
commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

3. Claims 1-3, 7, 15-17, 19-27, 29-32, 34, 37 and 48 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Demetri Terzopoulos, Xiaoyuan Tu, Radek Grzeszczukj 
"Artificial fishes: Autonomous locomotion, perception, behavior, and learning in a 
simulated physical world," December, 1994, Artificial Life, Vol. 1, No. 4, p. 327-351 
("Terzopoulos et al.") in view of U.S. Patent No. 6,600,491 to Szeliski et al. 
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4. With regard to claim 1, Terzopoulos et al. disclose "a method for on-screen animation of 
digital entities comprising: 

a. providing a digital world including image object elements (Plate 1c shows a 
digital world); 

b. providing at least one autonomous image entity (AIE) (2 nd paragraph of section 
7.2: "Each member of a school of artificial fish acts autonomously, and the schooling 
behavior is achieved through sensory perception and locomotion. "); 

c. each said AIE being associated with at least one AIE animation clip, and being 
characterized by a) attributes defining said at least one AIE relatively to said image 
objects elements (3 rd paragraph of section 6.2: " If the most dangerous predator is not 
too threatening (i.e. Fm < fl where fl > fO) and the fish has a schooling habit, then the 
school intention is generated, otherwise the escape intention is generated. "), and b) at 
least one behaviour for modifying at least one of said attributes {2 nd paragraph of section 
7.2: "Each fish constantly adjusts its speed and direction to match those of other 
members of the school. "; 2 nd paragraph of section 7.2: "Once a fish is in some proximity 
to some other schooling fish, the schooling behavior routine outlined in Fig. 16 is 
invoked. "); 

d. said at least one AIE including at least one virtual sensor for gathering data 
information about at least one of said image object elements or other one of said at least 
one AIE (Figure 10 shows an "Artificial fish vision sensor. "; 2 nd paragraph of section 
5.1: "In addition, the vision sensor can interrogate the world model database to identify 
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nearby objects and interrogate the physical simulation to obtain information such as the 
instantaneous positions and velocities of objects of interest. "); 

e. initializing said attributes and selecting one of said behaviours for each of said at 
least one AIE (2 nd paragraph of section 7.2: "An inceptive school is formed when a few 
fish swim towards a lead fish. "); 

f. for each said at least one AIE: 

i. using said at least one sensor to gather data information about at least one 
of said image object elements or other one of said at least one AIE {2 nd paragraph 
of section 7.2: "Each member of a school of artificial fish acts autonomously, and 
the schooling behavior is achieved through sensory perception and locomotion. "); 
and 

ii. using a decision tree for processing said data information resulting in at 
least one of i) triggering one of said at least one AIE animation clip according to 
said attributes and selected one of said at least one behaviour, and ii) selecting one 
of said at least one behaviour {Figure 16 shows a decision tree for schooling 
behavior with animation clips such as swimming at various speeds; Figure 12 
shows a decision tree for selecting behaviors such as schooling). 

5. One of ordinary skill in the art would recognize the actions taken by the artificial fish is 
analogous to an "AIE animation clip" from Figure 2, which shows that the artificial fish are 
animated objects in a computer graphics system. 

6. Terzopoulos et al. does not disclose "said AIE being associated with at least one 
repeatable AIE animation clip defined by a memorized sequence of images representing a given 
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moment." Szeliski et al. disclose an animated character "being associated with at least one 
repeatable animation clip (lines 48-51 of column 18: "Compound loops may contain repeated 
instances of the same primitive loop,., ") defined by a memorized sequence of images 
representing a given moment" (lines 1-3 of column 2: "The video texture is synthesized from a 
finite set of images by rearranging (and possibly blending) original frames from a source 
video. "; lines 51-54 of column 2: "Further, the frames of the video sprite could be inserted into a 
previously derived background image (or frames of a background video) at a location dictated 
by a prescribed path of the object in the scene. "; lines 12-20 of column 26: "...the runner makes 
natural-looking transitions between the different gaits in the generated video. Thus, a kind of 
"parametric motion control" results. This could easily be extended to other kinds of variants on 
running (higher kick, uphill/downhill), or other movements (say a sequence of dance or martial 
arts steps). " ; see also lines 58-62 of column 26 describing a controlled path). 

7. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the "video textures" and associated teachings of a parametric motion control 
disclosed by Szeliski et al. to animate the fish in the system disclosed by Terzopoulos et al. The 
motivation for doing so would have been to animate complex objects and processes efficiently, 
as well as to utilize real-world objects to provide a more realistic animation as suggested by 
Szeliski et al. in lines 15-18 of column 2, and lines 6-19 of column 27. Therefore, it would have 
been obvious to combine Terzopoulos et al. with Szeliski et al. to obtain the invention specified 
in claim 1. 

8. With regard to claim 2, Terzopoulos et al. disclose "said at least one AIE being 
associated with a memory for storing said data information (1 st paragraph of section 6.3.* "If the 
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current behavior is interrupted by a high priority event, the intention generator stores in a 
single-item short term memory the current intention and some associated information that may 
be used to resume the interrupted behavior. "); said using a decision tree for processing said data 
information {Figure 12 shows a decision tree and the step f = eat or mate) resulting in at least 
one of i) triggering one of said at least one AIE animation clip according to said attributes and 
selected one of said at least one behaviour, ii) selecting one of said at least one behaviour (1 st 
paragraph of section 6.4; "Once the intention generator selects an intention it attempts to satisfy 
the intention by passing control to a behavior routine along with the data from the perception 
focusser. "; as previously shown Figure 16 shows animation clips according to said attributes), 
and iii) modifying said memory (Figure 12 shows "pop the memory" and "push the memory"). 
9. With regard to claim 3, Terzopoulos et al. disclose "a method further comprising: 

g. creating a group of AIEs (Figure 1 7 shows a group of AIEs); 

h. wherein said using a decision tree for processing said data information resulting in 
at least one of 

iii. triggering one of said at least one AIE animation clip according to said 
attributes and selected one of said at least one behaviour ii) selecting one of said 
at least one behaviour, (Figure 16 shows an animation clip for the schooling 
behavior; Figure 16 shows triggering the "swim at standard speed animation 
clip " according to the swimming direction attribute: "swim in generally the same 
direction as the neighbours? "), and 

iv. adding said at least one AIE to said group of AIEs (2 nd paragraph of 
section 7.2: "Once a fish is in some proximity to some other schooling fish the 
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schooling behavior routine outlined in Fig. 16 is invoked. "; 3 rd paragraph of 
section 7.2: " When a large school encounters an obstacle, the autonomous 
behavior of individual fishes trying to avoid the obstacle may cause the school to 
split into two groups and rejoin once the obstacle is cleared and the schooling 
behavior routine regains control. "). 

10. With regard to claim 7, Terzopoulos et al. disclose "said image object elements include 
two-dimensional or three-dimensional graphical representations of at least one of an object, a 
non-autonomous character, a building, a barrier, a terrain, and a surface," wherein Terzopoulos et 
al. discloses a barrier in Figure 10 (b). 

11. With regard to claim 15, Terzopoulos et al. disclose "said attributes include at least one 
internal state attribute defining a non-apparent characteristic of said at least one AIE {T* 
paragraph of section 6. 1: 'The artificial fish has three mental state variables, hunger H, libido 
L, and fear F. The range of each variable is [0; I], with higher values indicating a stronger urge 
to eat, mate and avoid danger, respectively. "); said at least one behaviour is a state change 
behaviour for modifying said at least one internal state attribute {3 rd paragraph of section 6.2: "If 
there is no immediate danger of collision, the neighborhood is searched for predators, the fear 
state variable F and the most dangerous predator m for which Fm > Fi are calculated. If the 
total fear F> fO (where 0:1 <fD <0:5 is a threshold value) evasive action is to be taken. "). 

12. With regard to claim 16, Terzopoulos et al. disclose "at least one behaviour is a 
locomotive behaviour for causing said at least one AIE to move" (Figure 12 shows a schooling 
behavior causing an AIE to i( make appropriate turns to match the general orientation of the 
neighbours'' and u speed up towards" the closest schoolmate). 
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13. With regard to claim 17, Terzopoulos et al. disclose "said at least one behaviour includes 
a plurality of behaviours (Figure 16 shows the schooling behavior comprises "find the closest 
schoolmate in front and speed up towards it, " "swim at the standard speed, " and "turning"); 
each of said plurality of behaviours producing a behavioural steering force defined by an 
intensity (Figure 4; 3 rd paragraph of section 3.4: "The pectoral fins (Fig. 5) work by applying 
reaction forces to nodes in the midsection, i.e. nodes 1 <i < 12 (see Fig. 3). "; 2 nd paragraph of 
section 3.3: "Both the swimming speed and the turn angle of the fish model are approximately 
proportional to the contraction amplitudes and frequencies/rates of the muscle springs. "; 3 rd 
paragraph of section 3.4: "Assuming that a fin has an area A, surface normal n and the fish has 
a velocity v relative to the water (Fig. 5), the fin force /.... "); whereby, in operation, each of said 
plurality of behaviours producing a steering force on said at least one AIE proportionate to said 
intensity" (3 rd paragraph of section 3.4: "This motion control is useful in maintaining schooling 
patterns, for instance. "; Figure 16 shows changing swimming speed and direction). 

14. With regard to claim 19, Szeliski et al. further discloses "a blend time defining a number 
of frame that said at least one movement take to change from an active state to an inactive state" 
(line 65 of column 21 to line 1 of column 22: "Instead of simply jumping from one frame to 
another when a transition is made, the images of the sequence before and after the transition can 
be blended together via conventional blending methods. "; lines 11-19 of column 22: "For 
example, referring to FIG. 14, the last three frames 1400-1402 of the video sequence prior to an 
unacceptable transition are respectively blended with the first three frames 1403-1405 of the 
video sequence after the transition. The ratio formula used dictates that last frame 1400 of the 
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prior video sequence accounts for one-quarter of the blended is frame 1406 with the third frame 

1405 of the subsequent sequence accounting for three-quarters of the blended frame. "). 

15. With regard to claim 37, Szeliski et al. further discloses "wherein in if one of said at least 
one animation clip associated to said at least one object plays before said at least one animation 
clip is triggered then said at least one animation clip is triggered and a blend animation is created 
between said one of said at least one animation clip associated to said at least one object playing 
before said at least one animation clip is triggered and said at least one animation clip" {lines 11- 
19 of column 22: "For example, referring to FIG. 14, the last three frames 1400-1402 of the 
video sequence prior to an unacceptable transition are respectively blended with the first three 
frames 1403-1405 of the video sequence after the transition. The ratio formula used dictates that 
last frame 1400 of the prior video sequence accounts for one-quarter of the blended is frame 

1406 with the third frame 1405 of the subsequent sequence accounting for three-quarters of the 
blended frame. "). 

16. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the three frame blend time for video textures as disclosed by Szeliski et al. to render 
and characterize the behaviors disclosed by Terzopoulos et al. The motivation for doing so 
would have been to provide a gradual and smooth transition as suggested by Szeliski et al. in 
lines 41-57 of column 21 . Therefore, it would have been obvious to further modify the 
combination of Szeliski et al. and Terzopoulos et al. to obtain the invention specified in claims 
19 and 37. 

17. With regard to claim 20, Terzopoulos et al. disclose "said at least one behavior is 
triggered based on one of the said AIE's attributes" (3 rd paragraph of section 6.2: " If the most 



Application/Control Number: Page 10 

10/772,028 

Art Unit: 2628 

dangerous predator is not too threatening (i.e. Fm < fl where fl > fO) and the fish has a 
schooling habit, then the school intention is generated, otherwise the escape intention is' 
generated. "; 1 st paragraph of section 6.4: "Once the intention generator selects an intention it 
attempts to satisfy the intention by passing control to a behavior routine along with the data from 
the perception focus ser. "). 

18. With regard to claim 21, Terzopoulos et al. disclose "said at least one behaviour is 
triggered based on a distance of said at least one AIE to one of said image object elements and 
another AIE" {3 rd paragraph of section 6.4: "The eating food routine tests the distance d from 
the fish 's mouth to the food (see Fig. 5). If d is greater than some threshold value, the subroutine 
chasing-target is invoked. When d is less than the threshold value the subroutine suck-in is 
activated where a "vacuum" force (to be explained in Sec. 6.1) is calculated and then exerted on 
the food. "; 2 nd paragraph of section 6.2: "A large sensitivity region results in a 'timid' fish that 
takes evasive action to avoid a potential collision well in advance, while a tight sensitivity region 
yields a 'courageous 'fish that takes evasive action only at the last second. ") 

19. With regard to claim 22, Terzopoulos et al. disclose "said at least one behaviour is 
characterized by an activation radius defining the minimal distance between said at least one AIE 
and said targeted one of said image object elements" (2 nd paragraph of section 6.2: "The 
intention generator first checks the sensory information stream to see if there is any immediate 
danger of collision. "; 1 st paragraph of section 5.1 (emphasis added): "The cyclopean vision 
sensor has a 300 degree spherical field of view extending frontally and latterally to an effective 
radius Vr appropriate to the visibility of the translucent water (Fig. 10(a)). An object is "seen " 
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only if some part of it enters this view volume and it is not fully occluded behind some other 
opaque object (Fig. 10(b)). "). 

20. With regard to claim 23 , Terzopoulos et al. disclose "wherein said at least one behaviour 
causes said at least one AIE to perform an action selected form the group consisting of: 

i. moving towards another AIE (T paragraph of section 7.2: "An inceptive school 

is formed when a few fish swim towards a lead fish. "); 

j. fleeing from another AIE; looking at another AIE (3 rd paragraph of section 6.4: 
"The escaping routine chooses a suitable MC according to the relative position, 
orientation of the predator to the fish. "); 

k. orbiting said targeted one of said image object elements or another AIE (2 nd 
paragraph of section 7.3: "Two frequently observed patterns are looping, in which the 
male swims vigorously up and down in a loop slightly above and in front of the female, 
and circling, in which the male and female circle, seemingly chasing each other 's tail. "; 
2nd paragraph of section 7.3: "We have implemented a reasonably elaborate courtship 
behavior routine which simulates courtship dancing, circling, spawning ascent, and 
nuzzling behavior patterns in sequence (Plate lb). "); 

1. aligning with at least one another AIE (Figure 12 shows "make appropriate turns 
to match the general orienation of the neighbours"); 

m. joining with at least one another AIE (3 rd paragraph of section 7.2: "When a large 
school encounters an obstacle, the autonomous behavior of individual fishes trying to 
avoid the obstacle may cause the school to split into two groups and rejoin once the 
obstacle is cleared and the schooling behavior routine regains control fr )\ and 
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n. keeping a distance to at least one another AIE M {2 nd paragraph of section 7.2 
(emphasis): 'They establish a certain distance from one another, roughly one body length 
from neighbors, on average [25], Each member of a school of artificial fish acts 
autonomously, and the schooling behavior is achieved through sensory perception and 
locomotion. "). 

2 1 . With regard to claim 24, Terzopoulos et al. disclose "at least one behaviour further 
modifying a targeted one of said image object elements or another AIE" {3 rd paragraph of 
section 6.2: "If there is no immediate danger of collision, the neighborhood is searched for 
predators, the fear state variable F and the most dangerous predator m for which Fm >Fi are 
calculated. "; 7 paragraph of section 7.3: 11 It is intriguing to watch some of the male artificial 
fish 's nuzzling attempts fail because of an inappropriate approach angle to the female which 
triggers the avoiding-fish response. The male turns away to avoid the collision and tries 
again. "). 

22. With regard to claim 25, Terzopoulos et al. disclose "said at least one behaviour causes 
said at least one AIE to perform an action selected form the group consisting of: 

o. avoiding one of said image object elements or another AIE {7 th paragraph of 
section 7.3 (emphasis added): "It is intriguing to watch some of the male artificial fish 's 
nuzzling attempts fail because of an inappropriate approach angle to the female which 
triggers the avoiding-fish response . "); 

p. accelerating said at least one AIE (Figure 16 shows "find the closest schoolmate 
in front and speed up towards it") \ 

q. maintaining a constant speed (Figure 16 shows u swim at the standard speed"); 
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r. moving randomly within a selected portion of said digital world (4 th paragraph of 
section 6.4: "The wandering routine sets the fish swimming at a certain speed by invoking 
the swim-MC, while sending random turn angles to the turn-MCs. "); and 
s. attempting to face a predetermined direction (Figure 16 shows "make appropriate 
turns to match the general orientation of the neighbours "). 

23. With regard to claim 26, Terzopoulos et al. disclose "at least one virtual sensor is a vision 
sensor (Figure 10 shows an "Artificial fish vision sensor, ") for detecting said at least one of said 
image object elements or another one of said at least one AIE when said at least one of said 
image object elements or another one of said at least one AIE is within a predetermined distance 
from said at least one AIE and within a predetermined frustum issued therefrom (1 st paragraph 
of section 5.1: "The cyclopean vision sensor has a 300 degree spherical field of view extending 
frontally and latterally to an effective radius Vr appropriate to the visibility of the translucent 
water (Fig. 10(a)). An object is "seen " only if some part of it enters this view volume and it is 
not fully occluded behind some other opaque object (Fig. 10(b)). "). 

24. With regard to claim 27, Terzopoulos et al. disclose "said at least one virtual sensor is a 
property sensor for detecting at least one attribute of said other one of said at least one AIE" (2 nd 
paragraph of section 5.1: "In addition, the vision sensor can interrogate the world model 
database to identify nearby objects and interrogate the physical simulation to obtain information 
such as the instantaneous positions and velocities of objects of interest. "; 3 rd paragraph of 
section 6.4: "The eating-food routine tests the distance d from the fish f s mouth to the food (see 
Fig. 5), If d is greater than some threshold value, the subroutine chasing-target is invoked. "; 4 
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paragraph of section 6.4: "The escaping routine chooses a suitable MC according to the relative 
position, orientation of the predator to the fish "). 

25. With regard to claim 29, Terzopoulos et al. disclose "said data information is stored in a 
datum" (3 rd paragraph of section 1.2: "It ensures that goals have some persistence by exploiting 
a single-item memory. "). Terzopoulos et al. does not use the language "datum"; however, one of 
ordinary skill in the art would recognize "a single-item memory" is analogous to a "datum" as 
recited in claim 29. 

26. With regard to claim 30, Terzopoulos et al. disclose "said virtual sensor allows for setting 
a value stored in said datum based on one of said attributes" (Figure 12 shows "F™ < f? and 
likes schooling" then / = escape else / = school). 

27. With regard to claim 31, Terzopoulos et al. disclose "said virtual sensor allows for setting 
a value stored in said datum based on whether or not a predetermined one of said at least one 
AIE animation clip is triggered" (7 th paragraph of section 7.3: "The nuzzling routine requires the 
male to approach her abdomen from below... It is intriguing to watch some of the male artificial 
fish *s nuzzling attempts fail because of an inappropriate approach angle to the female which 
triggers the avoiding-fish response. "). One of ordinary skill in the art would recognize the 
"nuzzling routine" is analogous to an animation clip as recited in claim 31 from figure 2, which 
shows that the artificial fish are animated objects in a computer graphics system. 

28. With regard to claim 32, Terzopoulos et al. disclose "said at least one AIE animation clip 
is triggered after an active animation associated to said at least one AIE is completed" (3 rd 
paragraph of section 6.4: "The eating-food routine tests the distance d from the fish 's mouth to 
the food (see Fig. 5). If d is greater than some threshold value, the subroutine chasing-target is 
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invoked. When d is less than the threshold value the subroutine suck-in is activated where a 
"vacuum" force (to be explained in Sec. 6.1) is calculated and then exerted on the food"). 

29. With regard to claim 34, Terzopoulos et al. disclose "said attributes include the speed of 
said at least one AIE; in i) said at least one AIE animation clip being played at a speed depending 
on said speed of said at least one AIE" (3 rd paragraph of section 3.3: "The swim-MC (swim- 
MC(speed) {rj, si, r 2 , S2}) converts a swim speed parameter into contraction amplitude and 
frequency control parameters for the anterior (rl, si) and posterior (r2, s2) swim segments. "). 
One of ordinary skill in the art would recognize that the swimming motion of the artificial fish is 
analogous to an animation clip from the first paragraph of section 3.2: "The brain of the fish 
controls the muscles continuously through time by specifying the vector of muscle actuation 
functions u(t) = [uj(t),.. .,u 12(f)], whose components specify a time-varying contraction factor for 
each of the 12 muscles." 

30. With regard to claim 48, Terzopoulos et al. disclose "an artificial intelligence agent for 
on-screen animation of digital entities comprising: means to associate to an AIE a) attributes 
defining said AIE relatively to said image objects elements, (3 rd paragraph of section 6.2: 11 If 
the most dangerous predator is not too threatening (i.e. Fm < fl where fl > fO) and the fish has 
a schooling habit, then the school intention is generated, otherwise the escape intention is 
generated. ") b) a behaviour for modifying at least one of said attributes (2 nd paragraph of 
section 7.2: "Each fish constantly adjusts its speed and direction to match those of other 
members of the school. "; 2 nd paragraph of section 7.2: "Once a fish is in some proximity to some 
other schooling fish, the schooling behavior routine outlined in Fig. 16 is invoked. "), c) at least 
one virtual sensor for gathering data information about at least one of said image object elements 
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or other AIEs (Figure 10 shows an "Artificial fish vision sensor. "/ 2 nd paragraph of section 5.1: 
"In addition, the vision sensor can interrogate the world model database to identify nearby 
objects and interrogate the physical simulation to obtain information such as the instantaneous 
positions and velocities of objects of interest. "), and d) an AIE animation clips (1 st paragraph of 
section 6.4: "The artificial fish currently includes eight behavior routines: avoiding-static- 
obstacle, avoiding-fish, eating-food, mating, leaving, wandering, escaping, and schooling which 
serve the obvious purposes. The behavior routine uses the focused perceptual data to select an 
MC and provide it with the proper motor control parameters. "); and an autonomous image entity 
engine (AIEE) for updating each AIE's attributes and for triggering for each AIE at least one of a 
current behaviour and one of said at least one animation clip based on said current behaviour and 
said data information gathered by said at least one sensor (Figure 12 shows an AIEE). 
3 1 . Terzopoulos et al. does not disclose "d) at least one repeatable AIE animation clip, a 
repeatable AIE animation clip being defined by a memorized sequence of images representing a 
given moment." Szeliski et al. discloses "a repeatable animation clip (lines 48-51 of column 18: 
"Compound loops may contain repeated instances of the same primitive loop... ") being defined 
by a memorized sequence of images representing a given movement (lines 1-3 of column 2: "The 
video texture is synthesized from a finite set of images by rearranging (and possibly blending) 
original frames from a source video. "; lines 51-54 of column 2: "Further, the frames of the video 
sprite could be inserted into a previously derived background image (or frames of a background 
video) at a location dictated by a prescribed path of the object in the scene. "; lines 12-20 of 
column 26: " ...the runner makes natural-looking transitions between the different gaits in the 
generated video. Thus, a kind of "parametric motion control" results. This could easily be 
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extended to other kinds of variants on running (higher kick uphill/downhill), or other movements 
(say a sequence of dance or martial arts steps). " ; see also lines 58-62 of column 26 describing a 
controlled path). 

32. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use "video textures" and associated teachings of a parametric motion control and 
controlled path disclosed by Szeliski et al. to animate the fish in the system disclosed by 
Terzopoulos et al. The motivation for doing so would have been to animate complex objects and 
processes efficiently, as well as to utilize real-world objects to provide a more realistic animation 
as suggested by Szeliski et al. in lines 15-18 of column 2. Therefore, it would have been obvious 
to combine Terzopoulos et al. with Szeliski et al. to obtain the invention specified in claim 48. 

33. Claims 1, 5, 6, 7, 13, 18, 45, 46, and 49 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hansrudi Noser, Olivier Renault, Daniel Thalmann, Nadia Magnenat- 
Thalmann, "Navigation for Digital Actors based on Synthetic Vision, Memory, and 
Learning," 1995, Computers & Graphics Vol. 19, No. 1, p. 7-19 (Noser et al) in view of U.S. 
Patent No. 6,600,491 to Szeliski et al. 

34. With regard to claim 1, Noser et al. discloses "a method for on-screen animation of 
digital entities comprising: 

t. providing a digital world including image object elements {Figures 5, 8 and 9 on 
pages 12 and 13 show world with corridors and actors) 

u. providing at least one autonomous image entity (AIE) (1 st paragraph of section 
4.2: "The three basic modules of the local navigation are integrated in an interactive 
program allowing a user to perform simulation with digital actors in real time. "); 
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v. each said AIE being associated with at least one AIE animation clip, and being 
characterized by a) attributes defining said at least one AIE relatively to said image 
objects elements (1 st paragraph of section 3.4: " The path finding procedure described 
above is a mental process of the actor, which is based on the contents of his visual 
memory (octree). "), and b) at least one behaviour for modifying at least one of said 
attributes {1 st paragraph of section 3.4: "This exploring is an active process and the actor 
has to walk and memorize what he sees. "); 

w. said at least one AIE including at least one virtual sensor for gathering data 
information about at least one of said image object elements or other one of said at least 
one AIE (1 st paragraph of section 1.3: "The originality of our approach is the use of a 
synthetic vision as a main information channel between the environment and the digital 
actor. "); 

x. initializing said attributes {section 3.2: "After each deletion process, the octree X 
is reinitialized. ") and selecting one of said behaviours for each of said at least one AIE ( 
section 3.3: "We can create a new octree containing only the voxels of the path and their 
neighbors according to a given heuristic. This extended path octree(pathOctree) 
corresponds to the topology of a real road (2D) or channel (3D), where the actor can 
displace himself"; 1 st paragraph of section 4.1.3: "This module contains the DLAs. 
There are three families of DLA : the DLAs creating the global goal (follow the _corridor, 
follow Jhejwall, follow Jhejvisualjguide), the DLAs creating the local goal 
(avoid _obstacle, closest to _goal), and the DLAs effectively moving the actor 
(go Jo _global _goal). "); 
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y. for each said at least one AIE: 

v. using said at least one sensor to gather data information about at least one 
of said image object elements or other one of said at least one AIE {1 st paragraph 
of section 4.1: "The vision module has a modified version of the drawing routine 
traveling the world; instead of giving the real color of the object to the graphic 
engine, this routine gives a code, call the vision jd, which is unique for each 
object and actor in the world. This code allows the image recognition and 
interpretation. "); and 

vi. using a decision tree for processing said data information {section 4. 1.2 on 
page 12: "If local goal \ move to the local _goal \ else if special status from the 
DLA \ take decision \ else if global _goal \ move to global goal ") resulting in at 
least one of i) triggering one of said at least one AIE animation clip according to 
said attributes and selected one of said at least one behaviour, and ii) selecting one 
of said at least one behaviour {1 st paragraph of section 4.1.3: "This module 
contains the DLAs. There are three families of DLA: the DLAs creating the global 
goal (follow Jhejcorridor, follow _the_wall, follow Jhejyisual_guide), the DLAs 
creating the local goal (avoid_obstacle, closest Jo _goal), and the DLAs 
effectively moving the actor (go_to_globalj*oal). n ; 4 th paragraph of section 4.2: 
"We have implemented "guides" the animator can move while recording the 
motion. These guides are goals for the actor which try to reach them, avoiding 
obstacles. "). 
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35. One of ordinary skill in the art would recognize that "follow_the_corridor, 
follow_the_wall, follow_the_visual_guide" are analogous to behaviors. One of ordinary -skill in 
the art would recognize from the fourth paragraph of section 4.2 that an animation clip of the 
actor moving through the virtual environment is provided to the animator. Furthermore, one of 
ordinary skill in the art would recognize the nested if conditional statements on page 12 are 
analogous to a decision tree. 

36. Noser et al. does not disclose "said AIE being associated with at least one repeatable AIE 
animation clip defined by a memorized sequence of images representing a given moment." 
Szeliski et al. disclose a character "being associated with at least one repeatable animation clip 
(lines 48-51 of column 18: "Compound loops may contain repeated instances of the same 
primitive loop... ") defined by a memorized sequence of images representing a given moment" 
(lines 1-3 of column 2: "The video texture is synthesized from a finite set of images by 
rearranging (and possibly blending) original frames from a source video. "; lines 51-54 of 
column 2: "Further, the frames of the video sprite could be inserted into a previously derived 
background image (or frames of a background video) at a location dictated by a prescribed path 
of the object in the scene. "; lines 12-20 of column 26: " ...the runner makes natural-looking 
transitions between the different gaits in the generated video. Thus, a kind of "parametric motion 
control" results. This could easily be extended to other kinds of variants on running (higher kick, 
uphill/downhill), or other movements (say a sequence of dance or martial arts steps). " ; see also 
lines 58-62 of column 26 describing a controlled path). 

37. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the "video textures" disclosed by Szeliski et al. to animate the actor disclosed by 
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Noser et al. The motivation for doing so would have been to animate complex objects and 
processes efficiently, as well as to utilize real-world objects to provide a more realistic animation 
as suggested by Szeliski et al. in lines 15-18 of column 2. Therefore, it would have been obvious 
to combine Noser et al. with Szeliski et al. to obtain the invention specified in claim 1 . . 

38. With regard to claim 5, Noser et al. discloses "said image object elements include two- 
dimensional or three-dimensional graphical representations of a surface (Figure 4 shows image 
object elements representing walls of a maze); said attributes including at least one of: an 
attribute defining whether or not said least one AIE hugs said surface (Figure 2 shows hugging a 
surface; 1 st paragraph of section 3.3: "So, the octree represents a graph with nodes and 
edges. The algorithm of path searching uses the principle of backtracking and memorizes all 
tested nodes in a sorted list. With this list of already tested nodes, circuits can be avoided, and 
situations without a path from a given source to a given destination can be detected. In a first 
approach, a path is represented by a sequence of free nodes")\ an attribute allowing setting 
whether or not said least one AIE aligns with a normal of said surface; and an attribute defining 
an extra height given to said at least one AIE relatively to said surface when said at least one AIE 
hugs said surface." One of ordinary skill in the art would recognize that the attributes of the AIE 
determine the path through the environment, and therefore whether or not said at least one AIE 
hugs said surface as shown in Figure 2 is determined by the octree from the statement in section 
3.4: "The path finding procedure described above is a mental process of the actor, which is based 
on the contents of his visual memory (octree)." 

39. With regard to claim 6, Noser et al. discloses "said surface is a barrier" (Figure 4 shows a 
maze comprising barriers navigated by the AIE; 4 th paragraph of section 3. 6: "In the maze there 
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was a wall which disappeared after some time. When the actor passed there a second time, he 
deleted it from his memory, too (see Figure 4). "). 

40. With regard to claim 7, Noser et al. discloses "said image object elements include two- 
dimensional or three-dimensional graphical representations of at least one of an object, a non- 
autonomous character, a building, a barrier, a terrain, and a surface," wherein Noser et al. 
discloses a barrier and objects in Figures 2, 5 and 6 and in section 3.6: "In the 2D search, an 
actor was placed in the interior of a maze with an impasse, a circuit and some animated flowers." 

41 . With regard to claim 1 3, Noser et al. discloses "at least one behaviour causes said at least 
one AIE to avoid said barrier" {Abstract (emphasis added): "His reasoning process allows him to 
find 3D paths based on his visual memory by avoiding impasses and circuits . "). 

42. With regard to claim 1 8, Noser et al. discloses "at least one behaviour includes a plurality 
of behaviours" (4 th paragraph of section 4.1: " There are three families ofDLA: the DLAs 
creating the global goal (follow jhe_corridor, follow Jhe wall, follow Jhe_yisual_guide), the 
DLAs creating the local goal (avoid ^obstacle, closest to _goal), and the DLAs effectively moving 
the actor (go_to_global_goal). "); each of said plurality of behaviours producing a behavioural 
steering force {10 th paragraph of section 4.1: 'Tor obstacle avoidance, as there is no internal 
representation of the world, the strategy is the classic one: try to go straight to the global goal 
and if there is an obstacle in the direction, move around until there is a possible straight path to 
the global goal ") and being assigned a priority {2 nd paragraph of section 5: "The local ' 
navigation model uses the direct input information from his visual environment to reach goals or 
subgoals and to avoid unexpected obstacles. The goals or subgoals can be given by the global 
navigation module or directly by an interactive user system. "); whereby, in operation, each of 
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said plurality of behaviours being assigned to said at least one AIE by descending priority (3 rd 
paragraph of section 4.1: "The global goal, or final goal, is the goal the actor must reach. The 
local goal or temporary goal is the goal the actor creates to avoid the obstacles encountered in 
the path towards the global goal "). One of ordinary skill in the art would recognize local, 
temporary or sub-goals have a higher priority than the global goals. 

43. With regard to claim 45, Noser et al. discloses "a system for on-screen animation of 

digital entities comprising: 

z. an art package to create a digital world including image object elements and at 
least one autonomous image entity (AIE) and to create AIE animation clips {section 4.2: 
"The interface has been built using the Fifth Dimension Toolkit, an object-oriented user- 
interface toolkit developed in our lab. The world is described using a 3D data hierarchy 
developed for the actor's skeleton used for the walk... In this case the user should have 
some possibilities to direct the actor at a high level "); 
aa. an artificial intelligence agent to associate to an AIE 

vii. attributes defining said AIE relatively to said image objects elements (1 st 
paragraph of section 3.4: " The path finding procedure described above is a 
mental process of the actor, which is based on the contents of his visual memory 
(octree). "), 

viii. a behaviour for modifying at least one of said attributes (1 st paragraph of 
section 3.4: "This exploring is an active process and the actor has to walk and 
memorize what he sees. "), 
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ix. at least one virtual sensor for gathering data information about at least one 
of said image object elements or other AIEs (1 st paragraph of section 4.1: "The 
vision module has a modified version of the drawing routine traveling the world; 
instead of giving the real color of the object to the graphic engine, this routine 
gives a code, call the vision Jd, which is unique for each object and actor in the 
world. This code allows the image recognition and interpretation. "), and 

x. an AIE animation clips {section 4.2: " The animator records the trajectory 
of the guide using theSpaceBall to control the movements, he then plays the guide 
and lets the actor follow it. "); 

bb. said artificial intelligence agent including an autonomous image entity engine 
(AIEE) for updating each AIE's attributes and for triggering for each AIE at least one of a 
current behaviour and one of said at least one animation clip based on said current 
behaviour and said data information gathered by said at least one sensor (1 st paragraph of 
section 4.1.3: "This module contains the DLAs. There are three families ofDLA: the 
DLAs creating the global goal (follow _the_corridor, follow Jhe_wall, 
follow _the_visualjguide), the DLAs creating the local goal (avoid ' obstacle, 
closest Jo _goal), and the DLAs effectively moving the actor (goJo_global_goql). "; 4 th 
paragraph of section 4.2: "We have implemented "guides" the animator can move while 
recording the motion. These guides are goals for the actor which try to reach them, 
avoiding obstacles. "). 
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44. One of ordinary skill in the art would recognize from the fourth paragraph of section 4.2 
that an animation clip of the actor moving through the virtual environment is provided to the 
animator. 

45. Noser et al. does not disclose "creating repeatable AIE animation clips, a repeatable 
animation clip being* defined by a memorized sequence of images representing a given 
movement." Szeliski et al. disclose "a repeatable animation clip {lines 48-51 of column 18: 
"Compound loops may contain repeated instances of the same primitive loop... ") being defined 
by a memorized sequence of images representing a given movement" {lines 1-3 of column 2: 
"The video texture is synthesized from a finite set of images by rearranging (and possibly 
blending) original frames from a source video. "; lines 51-54 of column 2: "Further, the frames of 
the video sprite could be inserted into a previously derived, background image (or frames of a 
background video) at a location dictated by a prescribed path of the object in the scene. "; lines 
12-20 of column 26: " ...the runner makes natural-looking transitions between the different gaits 
in the generated video. Thus, a kind of "parametric motion control" results. This could easily be 
extended to other kinds of variants on running (higher kick uphill/downhill), or other movements 
(say a sequence of dance or martial arts steps). " ; see also lines 58-62 of column 26 describing a 
controlled path). 

46. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the "video textures" disclosed by Szeliski et al. to animate the actor disclosed by 
Noser et al. The motivation for doing so would have been to animate complex objects and 
processes efficiently, as well as to utilize real-world objects to provide a more realistic animation 
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as suggested by Szeliski et al. in lines 15-18 of column 2. Therefore, it would have been obvious 
to combine Noser et al. with Szeliski et al. to obtain the invention specified in claim 45. 

47. With regard to claim 46, Noser et al. discloses "a user interface for displaying and editing 
at least one of said at least one AIE and said image object elements" (3 rd paragraph of section 
4.2: " The interface has been built using the Fifth Dimension Toolkit, an object-oriented user- 
interface toolkit developed in our lab [[22]]. The world is described using a 3D data hierarchy 
[[23]] developed for the actor's skeleton used for the walk. "; 5 th paragraph of section 4.2: " In 
this case the user should have some possibilities to direct the actor at a high level. The idea is to 
give the animator a way to easily create the rough trajectory of the actor. "). 

48. With regard to claim 49, Noser et al. discloses "a system for on-screen animation of 
digital entities comprising (section 4.2: "This program is implemented in theC language on 
Silicon Graphics 4D workstations... The interface has been built using the Fifth Dimension 
Toolkit, an object-oriented user-interface toolkit developed in our lab. The world is described 
using a 3D data hierarchy developed for the actor's skeleton used for the walk): 

cc. means for providing a digital world including image object elements (Figures 5, 8 
and 9 on pages 12 and 13 show world with corridors and actors); 

dd. means for providing at least one autonomous image entity (AIE) (1 st paragraph of 
section 4.2: "The three basic modules of the local navigation are integrated in an 
interactive program allowing a user to perform simulation with digital actors in real 
time. "); 

ee. each said AIE being associated with at least one AIE animation clip, and being 
characterized by a) attributes defining said at least one AIE relatively to said image 
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objects elements {1 st paragraph of section 3.4: " The path finding procedure described 
above is a mental process of the actor, which is based on the contents of his visual 
memory (octree). "), and b) at least one behaviour for modifying at least one of said 
attributes (1 st paragraph of section 3.4: "This exploring is an active process and the actor 
has to walk and memorize what he sees. "); 

ff. said at least one AIE including at least one virtual sensor for gathering data 
information about at least one of said image object elements or other one of said at least 
one AIE (1 st paragraph of section 1.3: "The originality of our approach is the use of a 
synthetic vision as a main information channel between the environment and the digital 
actor. "); 

gg. means for initializing said attributes (section 3.2: "After each deletion process, the 
octree X is reinitialized. ") and selecting one of said behaviours for each of said at least 
one AIE (section 3.3: "We can create a new octree containing only the voxels of the path 
and their neighbors according to a given heuristic. This extended path 
octree (pathOctree) corresponds to the topology of a real road (2D) or channel (3D), 
where the actor can displace himself "; 1 st paragraph of section 4. 1.3: "This module 
contains the DLAs. There are three families of DLA: the DLAs creating the global goal 
(follow Jhejcorridor, follow _the_wall, follow Jhe_yisual_guide), the DLAs creating the 
local goal (avoid_obstacle, closest to jgoal), and the DLAs effectively moving the actor 
(gojojglobal^goal)."); 

hh. means for using said at least one sensor to gather data information about at least 
one of the image object elements or other one of said each said at least one AIE (1 st 
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paragraph of section 4.1: "The vision module has a modified version of the drawing 
routine traveling the world; instead of giving the real color of the object to the graphic 
engine, this routine gives a code, call the visionjd, which is unique for each object and 
actor in the world. This code allows the image recognition and interpretation. "); 
ii. means for using a decision tree for processing said data information {section 4.1.2 
on page 12: u If local goal \ move to the local _goal \ else if special status from the DLA \ 
take decision \ else if global _goal \ move to global goal. "); 

jj. means for triggering one of said at least one AIE animation clip according to said 
attributes and selected one of said at least one behaviour; and means for selecting one of 
said at least one behaviour {1 st paragraph of section 4.1.3: "This module contains the 
. DLAs. There are three families of DLA: the DLAs creating the global goal 
(follow jhe_corridor, follow Jhejwall, follow the _visual_guide), the DLAs creating the 
local goal (avoid ' obstacle, closest Jo_goal), and the DLAs effectively moving the actor 
(go_to_globaljgoal). "; 4 th paragraph of section 4.2: "We have implemented "guides" the 
animator can move while recording the motion. These guides are goals for the actor 
which try to reach them, avoiding obstacles. "). 
49. Noser et al. does not disclose "each said AIE being associated with at least one repeatable 
AIE animation clip defined by a memorized sequence of images representing a given moment." 
Szeliski et al. discloses a character "being associated with at least one repeatable animation clip 
(lines 48-51 of column 18: "Compound loops may contain repeated instances of the same 
primitive loop... ") defined by a memorized sequence of images representing a given moment" 
(lines 1-3 of column 2: "The video texture is synthesized from a finite set of images by 
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rearranging (and possibly blending) original frames from a source video. "; lines 51-54 of 
column 2: "Further, the frames of the video sprite could be inserted into a previously derived 
background image (or frames of a background video) at a location dictated by a prescribed path 
of the object in the scene. "). 

50. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the "video textures" disclosed by Szeliski et aL to animate the actor disclosed by 
Noser et al. The motivation for doing so would have been to animate complex objects and 
processes efficiently, as well as to utilize real-world objects to provide a realistic animation as 
suggested by Szeliski et al. in lines 15-18 of column 2. Therefore, it would have been obvious to 
combine Noser et al. with Szeliski et al. to obtain the invention specified in claim 49. 

5 1 . Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Demetri 
Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial fishes: Autonomous locomotion, 
perception, behavior, and learning in a simulated physical world," December, 1994-, 
Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent No. 6,144,385 to Girard. 

52. With regard to claim 4, Terzopoulos et al. disclose "said attributes defining at least one 
AIE relatively to said image object elements include at least one of: . . .a maximum right turning 
angle of said at least one AIE; a maximum left turning angle of said at least one AIE. . "{6 th 
paragraph of section 3.3: "By interpolating the key parameters, we define a steering map that 
allows the fish to generate turns of approximately any angle up to 90 degrees. Turns greater 
than 90 degrees are composed as sequential turns of lesser angles. "). The combination of 
Terzopoulos et al. and Szeliski et al. does not define these attributes per frame. Girard discloses 
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defining animation attributes per frame (lines 45-47 of column 7 (emphasis added): "In timeline 
108, the duration that a foot is in contact with a given footstep is shown by the length of each 
footprint block shown as shaded squares along the timeline. In a preferred embodiment, the 
timeline is marked in terms of frames . "). 

53. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to define the attributes disclosed in Terzopoulos et al. per frame as disclosed by Girard. 
The motivation for doing so would have been provide the animator with greater degree of control 
over the animation. Therefore, it would have been obvious to combine Terzopoulos et al. and 
Szeliski et al. with Girard to obtain the invention specified in claim 4. 

54. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Demetri 
Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial fishes: Autonomous locomotion, 
perception, behavior, and learning in a simulated physical world," December, 1994, 
Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent Application Publication No. 2004/0036711 
to Anderson. 

55. With regard to claim 8, Terzopoulos et al. disclose direction vectors acting on the AIEs 
(3 rd paragraph of section 7.1: "To suck in prey, it opens its mouth and, while the mouth is open, 
exerts vacuum forces on fishes (the forces are added to external nodal forces fx in equation (1) 
and other dynamic particles in the vicinity of the open mouth, drawing them in (Fig. 14):") and a 
barrier in Figure 10 (b). The combination of Terzopoulos et al. and Szeliski et al. does not 
disclose a barrier defined by a forward direction vector. Anderson discloses "barrier is defined 
by a forward direction vector and is used to restrain the movement of at least one of said at least 
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one object in a direction opposite said forward direction vector" (paragraph [0020]: "Objects 
can also generate their own vectors to apply to other objects (e.g., a wall can generate a vector 
to discourage objects from penetrating the wall). "). 

56. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate barriers with forward direction vectors as taught by Anderson in the 
definition of barriers disclosed by Terzopoulos et al. The motivation for doing so would have 
been to add an additional dimension of realism as the force vector would enable artificial fish to 
interact with the barriers in a more realistic manner. Therefore, it would have been obvious to 
combine Anderson et aL with Terzopoulos et al. and Szeliski et al. to obtain the invention 
specified in claim 8. 

57. Claims 11 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Demetri Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, 11 Artificial fishes: Autonomous 
locomotion, perception, behavior, and learning in a simulated physical world," December, 
1994, Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent 
No. 6,600,491 to Szeliski et al. in view of U.S. Patent Application Publication No. 
2002/0060685 to Handley et al. 

58. With regard to claims 1 1 and 14, Terzopoulos et al. discloses a digital world and barriers 
(plates la-lb; Figure 9; Figure 10 (b))\ however, Terzopoulos et al. does not disclose "height- 
fields." Handley et al. discloses "said terrain are two-dimensional height-fields representation for 
bounding AIEs" and "said digital world is defined by parameters selected from the group 
consisting of a width, a depth, a height, and a center position" (paragraph [0008]: " Height fields 
defined on a pre-defined grid are often used to represent terrain. "). 
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59. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a height field defining a terrain as taught by Handley et al. in the system 
and method disclosed by Terzopoulos et al. The suggestion and motivation for doing so would 
have been to "increase realism and visual appeal" as suggested by Handley et al. in paragraph 
[0008]. Therefore, it would have been obvious to combine Handley et al. with Terzopoulos et al. 
and Szeliski et al. to obtain the invention specified in claims 1 1 and 14. 

60. Claims 9, 10 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hansrudi Noser, Olivier Renault, Daniel Thalmann, Nadia Magnenat-Thalmann, 
"Navigation for Digital Actors based on Synthetic Vision, Memory, and Learning," 1995, 
Computers & Graphics Vol. 19, No. 1, p. 7-19 ("Noser et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. 

61 . With regard to claims 9 and 12, Noser et al. disclose a three-dimensional barrier and a 
surface form three-dimensional shapes for constraining AIEs as shown in Figure 4, but does not 
expressly disclose "said barrier is defined by triangular planes" or "said surface includes 
triangular planes." Official Notice is taken that both the concept and the advantage of providing 
triangular planes for representing surfaces and barriers are well known and expected in the art. It 
would have been obvious to have included a triangular plane representation in Noser et al. as 
triangular planes are known to provide mathematical properties which make it a computationally 
efficient representation for many rendering algorithms. 

62. With regard to claim 10, Noser et al. disclose a three-dimensional barrier and a surface 
form three-dimensional shapes for constraining AIEs as shown in Figure 4, but does not 
expressly disclose "said barrier is a two-dimensional barrier defined by a line." Official Notice is 
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taken that both the concept and the advantage of providing two-dimensional lines for 
representing a barrier is well known and expected in the art. It would have been obvious to two- 
dimensional line representation in Noser et al. as two-dimensional lines are known to provide a 
demarcation of a boundary or territory without obstructing a user's view of the actor in virtual 
environments depicting a playing field for athletic competition (e.g., basketball, football, and 
soccer). 

63. Claim 28 is rejected under 35 U.S.C. 103(a) as being unpatentable over Demetri 
Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial fishes: Autonomous locomotion, 
perception, behavior, and learning in a simulated physical world," December, 1994, 
Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent No. 6,141,019 to Roseborough et al; 

64. With regard to claim 28, Terzopoulos et al. discloses a sensor in Figure 10; however, 
Terzopoulos et al. is silent with respect to a random sensor. Roseborough et al. discloses random 
selection for animating artificial creatures (lines 24-28 of column 15: " Upon beginning 
execution of a decision sub-behavior 80, a random selection is made based on a probability 
distribution given by the combination of the weighting factor 88 and evaluated to determine 
which sub-behavior 74 is executed. "). 

65. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a random sensor in the animation system disclosed by Terzopoulos* et al. 
The suggestion and motivation for doing so would have been to make the AIE behavior more 
realistic as suggested by Roseborough in lines 28-33 of column 15: "Using this very simple 
principle decision sub-behaviors 80 can accomplish quite complex results (e.g., personality). To 
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appreciate this concept one need merely appreciate that reality is random and, accordingly, 
modeling reality requires some randomness as an input." Therefore, it would have been obvious 
to combine Roseborough with Terzopoulos et al. and Szeliski et al. to obtain the invention 
specified in claim 28. 

66. Claim 35 is rejected under 35 U.S.C. 103(a) as being unpatentable over Demetri 
Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial fishes: Autonomous locomotion, 
perception, behavior, and learning in a simulated physical world," December, 1994, 
Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent No. 6,011,562 to Gagne et al. 

67. With regard to claim 35, Terzopoulos et al. and Szeliski et al. do not expressly disclose 
"one animation clip is scaled and a number of cycle is provided." Gagne et al. discloses "in i) 
said at least one animation clip is scaled and a number of cycle is provided for said at least one 
animation clip before said at least one animation clip is triggered" (lines 45-48 of column 9: 
"Specifically in a first mode, with cyclical Actions depicting repetitive events, the icon 
representing the Action can be resized to increase the total duration in which the Action is 
performed. "; lines 56-58 of column 9: "If lengthened, the object can perform more cycles than 
before. If shortened, the object can perform fewer cycles than before. "; lines 5-9 of column 10: 
"In a second mode, the Action icon can be re-sized to alter the speed with which an Action is 
performed. In the case of a cyclical Action, the object performs the same number of cycles as 
before, but over a longer or shorter period of time, effectively being a slower or faster 
animation. "). 
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68. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate scaling and providing the number of cycles for an animation clip as 
disclosed by Gagne et al. in the system and method for animation disclosed by Terzopoulos et al. 
and Szeliski et al. The motivation for doing so would have been so that a stored animation clip 
can be reused an made applicable a number of times thereby more efficiently utilizing 
computational resources. Therefore, it would have been obvious to combine Terzopoulos et al. 
and Szeliski et al. with Gagne et al. to obtain the invention specified in claim 35. 

69. Claim 36 is rejected under 35 U.S.C. 103(a) as being unpatentable over Demetri 
Terzopoulos, Xiaoyuan Tu, Radek Grzeszczuk, "Artificial fishes: Autonomous locomotion, 
perception, behavior, and learning in a simulated physical world," December, 1994, 
Artificial Life, Vol. 1, No. 4, p. 327-351 ("Terzopoulos et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent No. 6,208,357 to Koga et al. 

70. With regard to claim 36, Terzopoulos et al. discloses transition from one animation clip 
to another {5 th paragraph of section 7.3: "The male counts the number of times his mouth 
reaches the vicinity of the moving point, and when the count exceeds a set threshold (currently 6) 
he makes a transition from looping to circling behavior. "). Terzopoulos et al. does not expressly 
disclose an animation transition. Szeliski et al. discloses "a transition between clips" {line 65 of 
column 21 through line 1 of column 22: " Instead of simply jumping from one frame to another 
when a transition is made, the images of the sequence before and after the transition can be 
blended together via conventional blending methods. The second sequence is gradually blended 
into the first, while both sequences are running. FIG. 14 shows an example of this process, which 
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is called crossfading. "). Szeliski et al. does not expressly disclose an animation clip associated to 
said at least one object play before at least one animation clip is triggered. 

71 . Koga et al. discloses if one of said at least one animation clip associated to said at least 
one object plays before said at least one animation clip is triggered then playing an animation 
transition before said at least one animation clip is triggered {lines 12-15 of column 8: "These 
transition properties may include one or more scripts associated with each transition. > These 
scripts may cause various activities to occur upon exiting one state and entering another state. 
When a particular transition is highlighted, the various transition properties associated with the 
transition are displayed in Frame 76. "). 

72. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate playing an animation transition as taught by Koga et al. in the system and 
method disclosed by Terzopoulos et al. between animation clips of AIEs such as "looping" and 
"circling." The motivation for doing so would have been to give the animation a more natural 
appearance. Therefore, it would have been obvious to combine Koga et al. with Terzopoulos et 
al. and Szeliski et al. to obtain the invention specified in claim 36. 

73. Claims 33 and 38-44 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hansrudi Noser, Olivier Renault, Daniel Thalmann, Nadia Magnenat-Thalmann, 
"Navigation for Digital Actors based on Synthetic Vision, Memory, and Learning;" 1995, 
Computers & Graphics Vol 19, No. 1, p. 7-19 ("Noser et al.") in view of U.S. Patent No. 
6,600,491 to Szeliski et al. in view of U.S. Patent No. 6,144,385 to Girard. 

74. With regard to claim 33, Noser et al. does not expressly disclose "a number of frame that 
said at least one animation clip will take to perform." Girard discloses "a number of frame that 
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said at least one animation clip will take to perform is provided before said at least one animation 
clip is triggered" {lines 3-6 of column 10: "In the footprint blocks, the numbers in the center of 
each block show the duration that the character's respective foot is in contact with the associated 
footprint. For example, the character's right foot is in contact with footprint 8 for a total of 24 
frames since footprint block 168 corresponding to footprint 8 covers 24 vertical demarcations on 
the timeline. tr ). 

75. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the animation system of Noser et al. by incorporating the number of frames one 
animation clip will take to perform as taught by Girard. The motivation for doing so would have 
been to provide the animator with a greater degree of control over the animation than the method 
disclosed by Noser et al. Therefore, it would have been obvious to combine Noser et al. with 
Girard to obtain the invention specified in claim 33. 

76. With regard to claim 38, Noser et al. discloses "said digital world includes at least one 
marking for modifying at least one of said attributes and said at least one behaviour of said at 
least one AIE" (2 nd paragraph of section 4.1.3: "In follow _the_visual _guide the vision is used to 
find the guide, to estimate its position and transform it into local coordinates. This visual guide 
can be either a dummy object visualizing the guide, an object of the decor or another actor. "; 2 nd 
paragraph of section 4.2: "These guides are goals for the actor which try to reach them, 
avoiding obstacles. "). One of ordinary skill in the art would recognize the position of the actor is 
an attribute as recited in claim 38, and the markers affect the motion of the actors and ho.w the 
actor behave in the environment. However, the combination of Noser et al. and Szeliski et al. 
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does not disclose "modifying on contact," as Noser et al. uses the visual sensor to determine the 
behavior of the actor. 

77. Girard discloses "said digital world includes at least one marking for modifying on 
contact at least one of said attributes" (lines 65-67 of column 9: "Next, FIGS. 7-10 are discussed 
to illustrate an animation sequence showing a character walking in accordance with the 
placement of footprints on a surface. "; lines 3-6 of column 10: " For example, the character's 
right foot is in contact with footprint 8 for a total of 24 frames since footprint block 168 ' 
corresponding to footprint 8 covers 24 vertical demarcations on the timeline. "). 

78. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the animation system of Noser et al. by incorporating modifying on contact 
with a marker the attributes and behavior of the actor as taught by Girard. The motivation for 
doing so would have been to allow an animator to guide the actor with more precision than the 
method disclosed by Noser et al. Therefore, it would have been obvious to combine Noser et al. 
with Girard to obtain the invention specified in claim 38. 

79. With regard to claim 39, Girard does not expressly disclose "at least one marking is 
defined by a bounding sphere having a radius," as the markings are footprints. However, it would 
have been obvious to one of ordinary skill in the art to modify the shape of the markings to "a 
bounding sphere having a radius" to obtain a marking that would be best-suited to the type of 
AIE being animated. The motivation for using a bounding sphere with a radius would be to 
accommodate AIEs, which move by flying like birds or insects, instead of the human-like AIE 
disclosed by Girard. 
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80. Claim 40 is met by the combination of Noser et al., Szeliski et al. and Girard, wherein . 
Noser et al. discloses, "said digital world includes a plurality of linked markings defining a path" 
(Figure 9 shows visual guide markers that are linked and defining a path through the 
environment). 

81. Claim 41 is met by the combination of Noser et al., Szeliski et al. and Girard, wherein 
Noser et al. discloses "said at least one behaviour causes said at least one AIE to use said path to 
navigate within said world towards one of said image object elements" (2 nd paragraph of section 
4. 1.3: "In follow Jhe ^visual _guide the vision is used to find the guide, to estimate its position 
and transform it into local coordinates. This visual guide can be either a dummy object 
visualizing the guide, an object of the decor or another actor. "). 

82. Claim 42 is met by the combination of Noser et al., Szeliski et al. and Girard, wherein 
Noser et al. discloses "some of said plurality of markings are linked with edges so as to define a 
waypoint network {Figure 9 shows arrow heads which are linked); and edge between two of said 
plurality of linked markings allowing said at least one AIE to move between said two of said 
plurality of linked markings (Figure 9: "use of a visual guide to direct an actor in a house 
environment"). One of ordinary skill in the art would recognize the actor moves through the 
house environment by following path of visual guides through the environment. 

83. Claim 43 is met by the combination of Noser et al., Szeliski et al. and Girard, wherein 
Noser et al. discloses "two of said plurality of linked markings being consecutive" (Figure 9 
shows consecutive markings). 

84. Claim 44 is met by the combination of Noser et al., Szeliski et al. and Girard, wherein 
Noser et al. discloses "at least one behaviour causes said at least one AIE to use said waypoint 
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network to navigate within said world towards one of said image object elements" (Figure 9: 
"use of a visual guide to direct an actor in a house environment"; 4 th paragraph of section 4.2: 
"We have implemented "guides" the animator can move while recording the motion. These 
guides are goals for the actor which try to reach them, avoiding obstacles. The animator can 
choose between a visual guide, a dummy object that the actor follows using his vision...."). One 
of ordinary skill in the art would recognize that the house environment comprises image object 
elements from Figure 9. 

85. Claim 47 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hansrudi 
Noser, Olivier Renault, Daniel Thalmann, Nadia Magnenat-Thalmann, "Navigation for 
Digital Actors based on Synthetic Vision, Memory, and Learning," 1995, Computers & 
Graphics Vol. 19, No. 1, p. 7-19 ("Noser et aL") in view of U.S. Patent No. 6,600,491 to 
Szeliski et al. in view of U.S. Patent No. 5,710,894 to Maulsby et al. 

86. With regard to claim 47, Noser et al. discloses a plurality of AIEs (Figure 8), and a user 
interface (section 4.2: "Three basic modules of the local navigation are integrated in an 
interactive program allowing a user to perform simulation with digital actors in real time."). The 
combination of Noser et al. and Szeliski et al. does not disclose a duplicating tool. Maulsby et al. 
discloses "a duplicating tool to simultaneously edit a plurality of objects (lines 1-6 of column 
11: "In FIG. 3B, the object property editor window 92 indicates a set of object properties 
including Property 1, Property 2, and Property j for object O m . As discussed further below, a 
property can be copied to all members of a selected dynamic class ("jar") by dragging and 
dropping the property onto the jar. ") 



Application/Control Number; Page 41 

10/772,028 

Art Unit: 2628 

87. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a duplicating tool as disclosed by Maulsby et al. in the interface disclosed 
by Noser et al. to edit the AIEs disclosed by Noser et al. The motivation for doing so would have 
been to increase the animator's productivity. Therefore, it would have been obvious to combine 
Noser et al. and Szeliski et al. with Maulsby et al. to obtain the invention specified in claim 47. 

Response to A rguments 

88. Applicant's arguments filed 9/17/2007 have been fully considered but they are not 
persuasive. Applicant's description of the Szeliski et al. reference does not accurately 
characterize its teachings. Szeliski et al. discloses a method for transitioning between sets of 
video frames representing a plurality of given movements of an object. In lines 12-20 of column 
26, Szeliski et al. provides an example of animating a runner by transitioning between different 
gaits and other movements: 

It is noted that since the system attempts to find frames that form a smooth transition 
from one to the next, when the user selects frames of the input video associated with a 
different running pace, the runner makes natural-looking transitions between the 
different gaits in the generated video. Thus, a kind of "parametric motion control" 
results. This could easily be extended to other kinds of variants on running (higher kick, 
uphill/downhill), or other movements (say a sequence of dance or martial arts steps). 
Applicant's allegation that the "appearance would be dictated solely by the limited feed loop" 
and therefore would "destroy the function of the autonomous animated fishes" is unpersuasive 
because Szeliski 's "parametric motion control" cannot be reasonably characterized as limiting 
the animation methods disclosed by Noser et al. and Terzopoulos et al. Specifically, the 
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behaviors described in Noser et al. and Terzopoulos et al. modify the speed and direction of the 
actor, and the animation created by Szeliski's video sprites are able to represent such changes as 
evidenced by lines 6-19 of column 27 and lines 58-62 of column 26, as discussed below. 

Applicant's given movements are analogous to Szeliski's disclosed different gaits, 
"higher kick," dance steps, and martial arts steps. Furthermore, one of ordinary skill in the art 
would have recognized the applicability of transitioning between different types of movements in 
Szeliski et al. to transitioning between different types of swimming movements in Terzopoulos 
from lines 6-19 of column 27, where Szeliski et al. discloses selecting the appropriate frames for 
different types of swimming motions for fish: 

As described previously, this is important to avoid the possibility of the selected frames 
depicting the fish as having a swimming motion that appears too slow or too fast 
compared to the translation speed of the fish through the water In other words, tt might 
appear that the fish is swimming very fast but only moving a short distance in the scene, 
or that fish is swimming very slowly but moving a great distance in the scene. Thus, 
considering not only the smoothness of the transition between frames but also the velocity 
of the fish associated with those frames, the process of selecting frames and inserting the 
selected sprite frame at appropriate points along the desired path can be coordinated so 
that the fish's local deformation appears to match its translation across the scene in the 
new video. 

Therefore, Applicant's characterization that "the video footage loop would remain the same for 
any movement" is inaccurate in view of Szeliski's teaching of transitioning between different 
gaits of running, or other movements. 
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89. As shown, the differences in speed of given movements such as walking or swimming is 
accounted for in Szeliski's video sprites, but it should be emphasized that the behaviors disclosed 
by Terzopoulos et al. and Noser et al. vary the actor's path as well. For example, Applicant's 
attention is directed to paragraph of section 4.1.3 in Noser et al. or the second paragraph of 
section 7.2 in Terzopoulos. Szeliski's video sprites are also able to accommodate this 
requirement: "For example, the user could select points in a background image displayed on a 
computer monitor that depicts a fish tank by moving a cursor along a desired path that the fish is 
to swim in the new video animation" (lines 58-62 of column 26). Utilizing Szeliski's teaching of 
a controlled path in behavioral animation would have been obvious. Therefore, at the time of the 
invention it would have been one of ordinary skill in the art to employ Szeliski's video sprites to 
animate the actors in Noser et al. or Terzopoulos et al. without destroying the purpose of either 
application. 

Conclusion 

90. All claims are drawn to the same invention claimed in the application prior to the entry of 
the submission under 37 CFR 1.114 and could have been finally rejected on the grounds and art 
of record in the next Office action if they had been entered in the application prior to entry under 
37 CFR 1.114. Accordingly, THIS ACTION IS MADE FINAL even though it is a first action 
after the filing of a request for continued examination and the submission under 37 CFR 1.114. 
See MPEP § 706.07(b). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
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MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 
1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, 
will the statutory period for reply expire later than SIX MONTHS from the mailing date "of this 
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